Á¤º¸°úÇÐȸ ÄÄÇ»ÆÃÀÇ ½ÇÁ¦ ³í¹®Áö (KIISE Transactions on Computing Practices)
Current Result Document :
ÇѱÛÁ¦¸ñ(Korean Title) |
ºÐ·ù Á¤È®µµ Çâ»óÀ» À§ÇÑ ¼±ÅÃÀû ¸¶½ºÅ· ±â¹Ý Ãß°¡ »çÀü ÇнÀ ±â¹ý |
¿µ¹®Á¦¸ñ(English Title) |
Improving Classification Accuracy Using Further Pre-training Scheme with Selective Masking |
ÀúÀÚ(Author) |
±è¿ìÀç
±è¸¸¹è
Á¤Àιü
Woojae Kim
Manbae Kim
Inbum Jung
¼¼ö¹Î
±è³²±Ô
Sumin Seo
Namgyu Kim
|
¿ø¹®¼ö·Ïó(Citation) |
VOL 27 NO. 09 PP. 0428 ~ 0439 (2021. 09) |
Çѱ۳»¿ë (Korean Abstract) |
ÃÖ±Ù ¿©·¯ ÀÚ¿¬¾î ó¸® ºÐ¾ß¿¡¼ »çÀü ÇнÀ ¾ð¾î ¸ðµ¨ÀÎ BERT¸¦ È°¿ëÇÏ¿© ºÐ¼® °úÁ¦¿¡ ÃÖÀûÈµÈ ÅؽºÆ® Ç¥ÇöÀ» ÃßÃâÇÏ·Á´Â ¿¬±¸°¡ È°¹ßÇÏ°Ô ÀÌ·ç¾îÁö°í ÀÖ´Ù. ƯÈ÷ BERTÀÇ ÇнÀ ¹æ½Ä Áß ÇϳªÀÎ MLM(Masked Language Model)À» È°¿ëÇÏ¿© µµ¸ÞÀÎ Á¤º¸ ¶Ç´Â ºÐ¼® °úÁ¦ µ¥ÀÌÅ͸¦ Ãß°¡ »çÀü ÇнÀ(Further Pre-training)ÇÏ´Â ½Ãµµ°¡ À̾îÁö°í ÀÖ´Ù. ÇÏÁö¸¸ ±âÁ¸ÀÇ MLM ±â¹ýÀÌ Ã¤ÅÃÇÑ ¹«ÀÛÀ§ ¸¶½ºÅ·À» »ç¿ëÇÏ¿© °¨¼º ºÐ·ù °úÁ¦¿¡¼ Ãß°¡ »çÀü ÇнÀÀ» ¼öÇàÇÏ´Â °æ¿ì, ºÐ·ù ÇнÀ¿¡ Áß¿äÇÑ ´Ü¼°¡ µÇ´Â ´Ü¾î°¡ ¸¶½ºÅ·µÉ ¼ö ÀÖ´Ù´Â °¡´É¼ºÀ¸·Î ÀÎÇØ ¹®Àå Àüü¿¡ ´ëÇÑ °¨¼º Á¤º¸ ÇнÀÀÌ ÃæºÐÈ÷ ÀÌ·ç¾îÁöÁö ¾Ê´Â´Ù´Â ÇÑ°è°¡ ÀÖ´Ù. ÀÌ¿¡ º» ¿¬±¸¿¡¼´Â ¹«ÀÛÀ§ ¸¶½ºÅ·ÀÌ ¾Æ´Ñ ´Ü¼ ´Ü¾î¸¦ Á¦¿ÜÇÏ°í ¸¶½ºÅ·ÇÏ´Â ¼±ÅÃÀû ¸¶½ºÅ·À» ÅëÇØ °¨¼º ºÐ·ù °úÁ¦¿¡ Æ¯ÈµÈ Ãß°¡ »çÀü ÇнÀÀ» ¼öÇàÇÒ ¼ö ÀÖ´Â ¹æ¹ýÀ» Á¦¾ÈÇÑ´Ù. ´õºÒ¾î ÁÖº¯ ´Ü¾î¸¦ ¼±ÅÃÇϱâ À§ÇØ ¾îÅÙ¼Ç ¸ÞÄ¿´ÏÁò(Attention Mechanism)À» È°¿ëÇÏ¿© ´Ü¾îÀÇ °¨¼º ±â¿©µµ¸¦ ÃøÁ¤ÇÏ´Â ¹æ¾Èµµ ÇÔ²² Á¦¾ÈÇÑ´Ù. Á¦¾È ¹æ¹ý·ÐÀ» ½ÇÁ¦ °¨¼º ´ñ±Û¿¡ Àû¿ëÇÏ¿© ¹®Àå º¤Å͸¦ Ãß·ÐÇÏ°í °¨¼º ºÐ·ù ½ÇÇèÀ» ¼öÇàÇÑ °á°ú, Á¦¾È ¹æ¹ý·ÐÀÌ ±âÁ¸ÀÇ ¿©·¯ ºñ±³ ¸ðµ¨¿¡ ºñÇØ ºÐ·ù Á¤È®µµ Ãø¸é¿¡¼ ¿ì¼öÇÑ ¼º´ÉÀ» ³ªÅ¸³¿À» È®ÀÎÇÏ¿´´Ù. |
¿µ¹®³»¿ë (English Abstract) |
Recently, studies to extract text expressions optimized for analysis tasks by utilizing bidirectional encoder representations from transformers (BERT), which is a pre-training language model, are being actively conducted in various natural language processing fields. In particular, attempts are being made to further pre-train domain information or target data using masked language model (MLM), which is one of the BERT training methods. However, if further pre-training is performed with the existing random masking when performing sentiment classification, there is a limitation that sentimental nuance for the entire sentence may not be sufficiently learned if the words that are important clues to the sentiment classification are masked. Therefore, in this study, we propose an further pre-training method specialized for sentiment classification tasks which sufficiently reflect sentiment information in sentences by selective masking that excludes clue words from masking candidates. In addition, this study proposes a method to distinguish between clue words and surrounding words as the role of words by utilizing the attention mechanism. On inferring sentence vectors by applying the proposed methodology to actual sentiment comments and performing sentiment classification experiments, it was confirmed that the proposed methodology showed superior performance in terms of classification accuracy compared to several existing comparison models. |
Å°¿öµå(Keyword) |
»ç¹°ÀÎÅͳÝ
¿§Áö ÄÄÇ»ÆÃ
¸ðÀǽÃÇè±â
°¡»ó »ç¹°
¿§Áö Ŭ·¯½ºÅÍ
internet of things
edge computing
simulator
virtual things
edge cluster
°¨¼º ºÐ¼®
BERT
MLM
¼±ÅÃÀû ¸¶½ºÅ·
¾îÅÙ¼Ç ¸ÞÄ¿´ÏÁò
sentiment analysis
BERT
MLM
selective masking
attention mechanism
|
ÆÄÀÏ÷ºÎ |
PDF ´Ù¿î·Îµå
|